Methods and apparatus for visualizing data

ABSTRACT

A method for visualizing data includes the step of providing a virtual flight visualization of image data representing a non-landscape object.

BACKGROUND OF THE INVENTION

This invention relates generally to a system for visualizing data, andmore particularly, to systems and methods that provide for a virtualflight visualization of a non-landscape object.

Typically, the visualization of 3-D medical scan data is performed usingsurface rendering or volume rendering. Surface rendering is well-suitedfor applications where segmentation of key structures is desired toprovide good visualization, such as with MRI data. Surface rendering isalso well-suited where a polygonal mesh can be wrapped around thesegmented structures to form a surface model. Volume rendering iswell-suited for applications where the assignment of color and opacityvalues are straight forward in light of the image voxel intensities.This is particularly true for CT data when there is a strong correlationbetween the Hounsfield units and tissue types.

Further, the rendering of medical scan data tends to be photo-realisticvolume rendering which can be augmented using non-photo-realisticrendering. For example, functional information, such as the fMRI of themotor cortex, is colorized and overlaid on gray-scale anatomical data.Further, 3-D graphical cues can be added to guide trajectories duringminimally invasive surgeries. Historically, the medical scan data hasbeen presented in such a manner that promotes geometric realism.

Therefore, it would be advantageous to provide systems and methods forvisualizing data which are not constrained by the need for geometricrealism, but rather can exaggerate contrast using not only color, butalso geometry.

BRIEF DESCRIPTION OF THE INVENTION

In one aspect, a method for visualizing data is provided wherein themethod includes the step of providing a virtual flight visualization ofimage data representing a non-landscape object.

In another aspect, a method of visualizing data includes the step ofassigning a height to a vertex based upon a non-landscape image pixelvalue or voxel value.

In yet another aspect, a system is provided. The system includes adetector positioned to receive energy transmitted through or emittedfrom a non-landscape object, a machine readable medium coupled to thedetector, and a set of machine readable instructions embodied in thecomputer readable medium for providing a virtual flight visualization ofthe received energy from the non-landscape object.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary diagnostic imaging system and resultingimage.

FIG. 2 illustrates an exemplary initial plot sheet.

FIG. 3 is a flowchart.

FIG. 4 illustrates a virtual flight visualization.

FIG. 5 illustrates a view from a point of reference of a plot sheet.

FIG. 6 illustrates exemplary view from a point of reference of a plotsheet.

FIG. 7 is a flowchart.

DETAILED DESCRIPTION OF THE INVENTION

There are herein described methods and apparatus useful for imagingsystems such as, for example, but not limited to an x-ray system. Theapparatus and methods are illustrated with reference to the figureswherein similar numbers indicate the same elements in all figures. Suchfigures are intended to be illustrative rather than limiting and areincluded herewith to facilitate explanation of an exemplary embodimentof the apparatus and methods of the invention. Although, described inthe setting of an x-ray system, it is contemplated that the benefits ofthe invention accrue to all diagnostic imaging systems and modalitiessuch as PET, MRI, SPECT, Ultrasound, fused systems such as a CT/PETsystem, and/or any modality yet to be developed in which data resultsfrom the imaging system.

FIG. 1 illustrates an imaging system 10 with an associated display 20.Imaging system 10 can be of any modality, but in one embodiment, system10 is a CT system. In another embodiment, system 10 is a dual modalityimaging system such as a combined CT/PET system and a scan can be donein one modality (e.g., CT) and the processed data can be transferred tothe other modality (e.g., PET). Display 20 can be separate from system10 or integrated with system 10. System 10 includes an acquisitiondevice such as an x-ray radiation detector, a Gamma Camera, anultrasound probe and/or an MRI coil.

The x-ray imaging system includes a processing circuit. The processingcircuit (e.g., a microcontroller, microprocessor, custom ASIC, or thelike) is coupled to a memory and a display device. The memory (e.g.,including one or more of a floppy disk drive, CD-ROM drive, DVD drive,magnetic optical disk (MOD) device, or any other digital deviceincluding a network connecting device such as an Ethernet device forreading instructions and/or data from a computer-readable medium, suchas a floppy disk, or an other digital source such as a network or theInternet, as well as yet to be developed digital means, and the like)stores imaging data.

The memory may also store a computer program including instructionsexecuted by the processing circuit to implement the functions describedherein. The processing circuit provides an image for display on adevice. The detector may be a flat panel solid state image detector, forexample, although conventional film images stored in digital form in thememory may also be processed. In one embodiment, the processing circuitexecutes instructions stored in firmware (not shown).

Once a scan is done using imaging system 10, image data 12 is producedwhich represents the results of the image. The image represents thehuman anatomy, or physical objects such as detected in airport or othertransportation scanners, or manufactured items such as semi-conductors.This image data does not represent topography of the earth's surface orother physical geographic regions such as those subject to map making,surveying, or graphical information systems. As such, the subject of theimage system is a non-landscape object.

Through the creation of the image from a scan, the herein describedmethods and apparatus can also be part of performing non-destructivetests on objects. For example, testing semi-conductors traditionally hasinvolved testing procedures that destroy the semi-conductor beingtested. Therefore, scanning processes, such as x-ray systems, have beenused to test semi-conductors in a non-destructive manner. The hereindescribed methods and apparatus can be used with such a testing processto provide a virtual flight simulation for non-destructive testing,including non-destructive testing of semi-conductors.

This non-landscape object is represented by image 12 which is the resultof the scan performed by the image scanner 10. A plot sheet 14 of FIG. 2is then created which has a vertex corresponding to a pixel of the imagedata. The image data is tessellated into a triangular mesh, in oneembodiment, so that the plot sheet contains a plurality of triangularportions. The scan is performed to create the image data in step 22 ofFIG. 3 and the initial plot sheet is created in step 24. Each vertex ofthe plot sheet is mapped to a pixel of the image data in step 26. Aheight is assigned to the vertex corresponding to the pixel based uponthe intensity of the corresponding pixel in step 28. A color is assignedto the vertex based upon the corresponding pixel of the image data instep 30. The color can be assigned based upon a function of brightnessor by anatomical information, or by functional information. It shouldalso be noted that the height for each vertex can be either a linear ornon-linear transfer function from the corresponding pixel. Therefore,the plot sheet can be displayed and has the advantage of showingcontrast between data points which are accentuated by both color andelevation. This provides for a novel perspective of displaying imagedata for non-landscape objects which provides advantages over simplydisplaying in color alone. The plot sheet can then be displayed in step32. Additionally, the image could be reformatted, such as super-sampledto create more pixels or sub-sampled to create fewer pixels, prior tomapping the image to the mesh. Also, the mesh need not be a triangularmesh, as squares would be another popular option. Any known flightcontrols, such as a joystick, may be used to achieve the visualization.

Once the plot sheet is displayed, the point of view and spatial frame ofreference can be altered to provide a virtual flight visualization 18 ofimage data 12 as shown in FIG. 4. In the virtual flight visualization,the user can view the plot sheet from any number of points of reference.For example, the user can manipulate the virtual flight visualization soas to make it appear that the user is viewing the plot sheet from pointsof reference 34 a-34 f, by way of example. Further, the user can “fly”along a path 35 to reach the points of reference thereby experiencingvirtual flight visualization. Further, the user can rotate the point ofreference view 360° in any direction with exemplary directions shown as36 a and 36 b.

When the user is viewing the plot sheet from point of reference 34 f,the view is shown in FIG. 5. This view is in generally an upwarddirection and shown for illustration purposes with a circular field ofview. When the user is viewing the plot sheet from point 34 a, the viewis shown in FIG. 6. This view is generally a downward direction andshown for illustrative purposes with a circular field of view. By movingthe point of reference, the user can fly around the plot sheet to viewthe plot sheet from a multitude of points about the plot sheet in threedimensions.

The point of reference can be controlled through machine readableinstructions to allow the user to explore the plot sheet area.Therefore, the user can manipulate the point of reference for differentpositions and orientations. Effectively, the user can pilot or fly thepoint of reference to achieve a desired view of the plot sheet area.Further, by manipulating the point of reference to within the plot sheetallowing the user to view certain areas while other areas are outsidethe field of view according to the point of reference.

Further, virtual flight visualization can allow the user to fly throughthe surfaces of the plot sheet, bounce off the surfaces, or fly under orthrough the surfaces to view the image from inside or outside the areaof the plot sheet.

The image can also have time points which represent slices of anon-landscape object image. Slices are sequences of sections through thenon-landscape object. The slices can each correspond to a plot sheet.Therefore, the plot sheets corresponding to the slices can be displayedin sequence so that the plot sheet may change over time. The user canthen see the vertex grow, shrink, or change color according to the plotsheet being displayed. As shown in FIG. 67, slices are created in step46 and if there are further slices to process in step 48, then the plotsheet is created for the unprocessed slices in step 50. The plot sheetis modified based upon steps 26 through 30 of FIG. 3 in step 52 and theprocess moves to the next layer in step 54. In another embodiment, theplot sheet can be modified without having to store a plot sheet for eachslice according to the particular slices. Therefore, the plot sheetvertex can grow, shrink, or change color according to the current slice.This allows the user to view changes in the plot sheet according to thecurrent slice.

Therefore, not only can the user have a virtual flight visualization fora non-landscape object for a particular time point, but can havemultiple time points to “fly” through. The plot sheet can be re-renderedaccording to each time point or slice to illustrate the changesaccording to the time points of the non-landscape object. As such, theheights and color of the plot sheet can be modified so that the plotsheet re-rendering represents the changes in the non-landscape objectover time.

In one embodiment, the user has the ability to easily switch backbetween regular mode and virtual flight mode. In another embodiment,where there may be a need to interpolate, when one has a prioriknowledge of the anatomy, one can use the a priori knowledge toeliminate or reduce false structures. Additionally, one can use ComputerAided Detection (CAD) programs with the virtual flight. Also, one canuse a computer aided diagnostic (also CAD) program to identify nodules,and then ignore all other structures to help eliminate or reduce falsepositives. Any known CAD computer assisted diagnostic may be used toidentify structure before doing the virtual flight, and aberrations maybe ignored.

Also one can use other artifact reduction techniques, for example, mosthuman anatomy is generally not streaky so one can ignore streaks whenfound in certain areas of the human body.

In general, artifact reduction can be built in with the flightsimulation based on one knowing the anatomy or other structure to bescanned. In other words, one may perform a CAD operation on the data andsuppress all features not detected by the CAD. One could also identifyregions just using a priori information on different regions or byassigning different base heights to different regions. For example in achest scan, the heart heights may vary from 1000 to 10000, while thelung heights vary from 0 to 9000, or one could have different dynamicranges for different structures, while the heart region can have 9000different values, maybe the lung region has 6000 or 20000, the dynamicrange can be optimized to aid processing speed versus storagerequirements.

Of course, the methods described herein are not limited to practice insystem 10 and can be utilized in connection with many other types andvariations of imaging systems. In one embodiment, the processing circuitis a computer that is programmed to perform functions described herein,and, as used herein, the term computer is not limited to just thoseintegrated circuits referred to in the art as computers, but broadlyrefers to computers, processors, microcontrollers, microcomputers,programmable logic controllers, application specific integratedcircuits, and other programmable circuits. Although the herein describedmethods are described in a human patient setting, it is contemplatedthat the benefits of the invention accrue to non-human imaging systemssuch as those systems typically employed in small animal research.Although the herein described methods are described in a medicalsetting, it is contemplated that the benefits of the invention accrue tonon-medical imaging systems such as those systems typically employed inan industrial setting, a transportation setting, such as, for example,but not limited to, a baggage scanning CT system for an airport or othertransportation center, or scanning manufactured items such assemi-conductor devices.

As used herein, an element or step recited in the singular and proceededwith the word “a” or “an” should be understood as not excluding pluralsaid elements or steps, unless such exclusion is explicitly recited.Furthermore, references to “one embodiment” of the present invention arenot intended to be interpreted as excluding the existence of additionalembodiments that also incorporate the recited features.

Also, as used herein, the phrase “reconstructing an image” is notintended to exclude embodiments of the present invention in which datarepresenting an image is generated but a viewable image is not.Therefore, as used herein the term, “image,” broadly refers to bothviewable images and data representing a viewable image. However, manyembodiments generate (or are configured to generate) at least oneviewable image.

Exemplary embodiments are described above in detail. The assemblies andmethods are not limited to the specific embodiments described herein,but rather, components of each assembly and/or method may be utilizedindependently and separately from other components described herein.

While the invention has been described in terms of various specificembodiments, those skilled in the art will recognize that the inventioncan be practiced with modification within the spirit and scope of theclaims.

1. A non-transitory computer readable medium having a computer programstored thereon and representing a set of instructions that when executedby a computer causes the computer to: acquire a set of image datacorresponding to a non-landscape object and comprising a plurality ofpixels; generate a series of three-dimensional plot sheets having aplurality of vertices corresponding to the plurality of pixels of theset of image data; assign a dimensional display characteristic to eachof the plurality of vertices; assign a non-dimensional displaycharacteristic to each of the plurality of vertices; and display theseries of three-dimensional plot sheets as a sequence of images, whereinthe dimensional and non-dimensional display characteristics of theplurality of vertices are re-rendered between sheets of the series ofthree-dimensional plot sheets; wherein the dimensional displaycharacteristic comprises a vertical height for each of the plurality ofvertices and the non-dimensional display characteristic comprises adisplay color for each of the plurality of vertices; and wherein thevertical height and the display color are assigned based on differentcriteria, with the vertical height being assigned to each of theplurality of vertices based on pixel intensity and the display colorbeing assigned to each of the plurality of vertices based on at leastone of anatomical information and functional information.
 2. Thenon-transitory computer readable medium of claim 1 wherein the set ofinstructions causes the computer to assign the vertical height to eachof the plurality of vertices based on an intensity of a respective pixelof the plurality of pixels.
 3. The non-transitory computer readablemedium of claim 1 wherein the set of instructions causes the computer todisplay a virtual flight visualization of the series ofthree-dimensional plot sheets responsive to a user input.
 4. Thenon-transitory computer readable medium of claim 3 wherein the set ofinstructions causes the computer to display a 360 degree reference viewat a user-selected point of reference of the series of three-dimensionalplot sheets.
 5. The non-transitory computer readable medium of claim 1wherein the set of instructions causes the computer to assign thedimensional and non-dimensional display characteristics using one of alinear transfer function and a non-linear transfer function.
 6. Thenon-transitory computer readable medium of claim 1 wherein the set ofinstructions causes the computer to generate a three-dimensional plotsheet having a plurality of triangular vertices.
 7. The non-transitorycomputer readable medium of claim 1 wherein the set of instructionsfurther causes the computer to re-render the three-dimensional plotsheet based on changes in the dimensional and non-dimensional displaycharacteristics over a plurality of time points.
 8. The non-transitorycomputer readable medium of claim 1 wherein the set of instructions thatcauses the computer to generate the series of three-dimensional plotsheets causes the computer to generate three-dimensional plot sheets torepresent respective slices of the non-landscape object.
 9. Acomputerized imaging method comprising: acquiring scan data from anon-landscape object using one of an x-ray system, a PET system, an MRIsystem, a SPECT system, an ultrasound system, and a CT/PET system;producing image data corresponding to the non-landscape object from thescan data from which an image of the non-landscape object can be formed;constructing a three-dimensional plot sheet having a plurality ofvertices corresponding to a plurality of pixels of the image data;generating a pair of display parameters for each of the plurality ofvertices based on the image data, wherein the pair of display parameterscomprises a dimensional parameter and a non-dimensional parameter, withthe dimensional parameter and the non-dimensional parameter beingdetermined based on separate and distinct image data parameters;displaying the pair of display parameters on the three-dimensional plotsheet; and generating a virtual flight visualization of thethree-dimensional plot sheet having a point of reference that isindependent of the acquisition of scan data.
 10. The imaging method ofclaim 9 comprising: generating a height for display of each of theplurality of vertices; and generating a color for display of each of theplurality of vertices.
 11. The imaging method of claim 9 furthercomprising enabling a user to manipulate the point of reference via auser interface to fly though surfaces of the three-dimensional plotsheet, bounce off surfaces of the three-dimensional plot sheet, and flyunder or through surfaces of the three-dimensional plot sheet.
 12. Theimaging method of claim 9 comprising accessing image data correspondingto one of an object of the human anatomy, a physical object, and amanufactured item.
 13. The imaging method of claim 9 further comprising:identifying regions of distinct anatomy on the three-dimensional plotsheet based on a priori information; and assigning a base height to theplurality of vertices within each respective region of distinct anatomy,wherein the base height for a given respective region of distinctanatomy has a magnitude corresponding to a type of distinct anatomy inthe given respective region.
 14. An imaging system comprising: anacquisition device configured to acquire image data corresponding to anon-landscape object, wherein the image data comprises a plurality ofpixels, and wherein the acquisition device comprises one of an x-raydetector, a Gamma Camera, an ultrasound probe, and an MRI coil; and aprocessor configured to: access the image data; generate athree-dimensional plot having a plurality of vertices, wherein eachvertex corresponds to a respective pixel of the image data; assigndisplay parameters to the plurality of vertices, one display parametergenerated from pixel intensity data and another display parameterassigned based on pixel functional information data; display athree-dimensional representation of the non-landscape object, whereinthe image data is represented as the display parameters on thethree-dimensional plot; display a virtual flight visualization of thethree-dimensional plot; and enable a user to manipulate a point ofreference of the virtual flight visualization at a point in timefollowing acquisition of the image data.
 15. The imaging system of claim14 wherein the processor is configured to assign a color representationand a height representation to each of the plurality of vertices. 16.The imaging system of claim 15 wherein the acquisition device comprisesan MRI system; and wherein the processor is further configured to:generate the height representation for display of each of the pluralityof vertices based on intensity data of a pixel corresponding to therespective vertex; and generate the color representation for display ofeach of the plurality of vertices based on functional MRI data (fMRI) ofa pixel corresponding to the respective vertex.
 17. The imaging systemof claim 14 wherein the user interface enables the user to simulateflight though surfaces of the three-dimensional plot, simulatedeflection off surfaces of the three-dimensional plot, and simulateflight under or around surfaces of the three-dimensional plot.
 18. Theimaging system of claim 14 wherein the processor is further configuredto: generate a series of three-dimensional plots; display the series ofthree-dimensional plots as a sequence of images; and enable the user toview changes in the display parameters across the sequence of images.